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ABSTRACT 


Today, biometric authentication plays vital role in the social and commercial area. This paper proposed a securing system access using multi-model biometrics for 
authorization with the embedded platform. It consists of the embedded system to verify the signatures, fingerprint and key pattern for authentication of user. Proposed 
system works in two phases of authentication i.e enrollment phase and evaluation phase. In enrollment phase all authentication factors are stored in data base and in 
evaluation phase stored one is compared with tested parameters. In proposed system HMM is implemented on android side for the signature, DTW is implemented on 
controller side for fingerprint and key pattern is with FPGA. Thus proposed work gives the two factor authentication biometric system with extra layer of security. 
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I. Introduction 

Day by day, natural and secure access to interconnected systems is becoming 
more and more important. Fingerprint and signature are the most pervasive meth- 
ods of individual identification and document authentication. Authentication of a 
person's identity is a need in many social and commercial interactions. It is also 
necessary to verify people identity in a fast, easy to use and user-friendly way. 
Because of this, biometric solutions are considered one of the most trusted and 
natural ways of identifying a person and controlling access to systems and appli- 
cations. On the basis of the input acquisition method, it can be categorized into 
static verification technique (offline) and dynamic verification technique (on- 
line). There are many more other personal authentication techniques as well. 
Some of them uses the possession of the token (i.e ID cards) and some of them are 
knowledge based (i.e password, key-phase etc). But, the token based technique 
whose attributes can be stolen or lost whereas knowledge based approaches 
whose attributes can be forgotten, which become major drawback of such tech- 
niques. But the biometrics attributes, do not suffer from such disadvantages. 

To the best of our knowledge, proposed biometric model gives the proper authen- 
tication with extra layer of security. We are used both traits of biometrics physio- 
logical trait fingerprint and behavioural trait signature. So, we say proposed sys- 
tem is a multi-model biometric system. Toward these direction the contribution 
of these paper is three fold: First we are using signature (behavioural trait), sec- 
ondly fingerprint (biological trait) and lastly key pattern. When all these three 
parameters matches then and only then user get authorized. Proposed system is 
used Two-factor authentication process. Proposed biometric authentication sys- 
tem built up by knowledge factor (key pattern) and inherence factor (signature 
and fingerprint). Proposed system is the dynamic verification technique. This 
paper is organized as follows. Section II describes the architecture of the 
multimodal biometric system. Section III presents working of proposed system. 
Section IV shows implementation of proposed system: Section V presents result 
and discussion and finally Section VI presents the conclusions. 

II. Architecture of Proposed System 

Fig. 1 shows the architecture adopt by proposed system which having generic 
architecture as a basic platform with some additional advantages features. Main 
building blocks of the architecture are listed below. As shown in fig 7. there are 
wireless connections in between Android and Matlab. Connection between 
android and Matlab is created through Wi-Fi. Matlab and microcontroller are con- 
nected through RS 232. Finger print module R302 is connected to 
microcontroller with serial port and microcontroller is connected to FPGA 
through 8 wire connection. 

A. Android phone 

We are using android phone as a data acquisition element for the user signature. 
In android, a touch gesture occurs when a user places one or more fingers on the 
touch screen, and application interprets that pattern of touches as a particular ges- 
ture. The gesture starts when the user first touches the screen, continues as the sys- 
tem tracks the position of the user's finger(s), and ends by capturing the final 
event of the user's fingers leaving the screen. Android provides the Gesture 
detector class for detecting common gestures. 

B. Finger print module R302 

This is a finger print sensor module with TTL UART interface for direct connec- 
tions to microcontroller UART or to PC through MAX232 / USB-Serial adapter. 
The user can store the finger print data in the module and can configure it in 1 : 1 or 
1 : N mode for identifying the person. 



Fig.l. Architecture of proposed Multimodal biometric system 

C. MATLAB 

MATLAB is a high- performance language for technical computing. It integrates 
computation, visualization and Programming in an easy-to-use environment 
where problems and solutions are expressed in familiar mathematical notation. 
MATLAB is complemented by family of application specific solutions called 
toolboxes. The signal Processing Toolbox is a collection of MATLAB functions 
(called M-functions or M-files) that extend the capability of the MATLAB envi- 
ronment for the solution of digital image processing problems 

D. Microcontroller (ATmegal62v-8PI) 

We are using microcontroller Atmega 162 V-8PI. It is a High-performance, Low- 
power AVR® 8-bit Microcontroller. The ATmegal62 is a low-power CMOS 8- 
bit microcontroller based on the AVR enhanced RISC architecture. By executing 
powerful instructions in a single clock cycle, the ATmegal62 achieves 
throughputs approaching 1 MIPS per MHz allowing the system designer to opti- 
mize power consumption versus processing speed. 

E. Key pattern 

In proposed system we are using key pattern as a third parameter of authentica- 
tion. Here we are implemented key pattern with help of FPGA. We are taking 
LSB as key pattern, so N number of users can be enrolled and authenticated by 
using proposed system. On the other hand, for the key pattern the last 3 bits of the 
ID will be used. Key pattern is based on the user ID. In proposed system, first 
three data pins (DO, D1 and D2) of microcontroller is for the in-code pattern 
whereas next three pins (D4,D5,D6) are used for key pattern and last pin D7 is 
used for showing the result of key pattern matching. If we get 1 in pin D7 it means 
key pattern is matched and if we get 0 as a output at pin D7 it means key pattern is 
not matched. 

F. FPGA 

A field-programmable gate array (FPGA) is an integrated circuit designed to be 
configured by a customer or a designer after manufacturing - hence "field- 
programmable". The FPGA configuration is generally specified using a hard- 
ware description language (HDL), similar to that used for an application-specific 
integrated circuit (ASIC). FPGAs contain an array of programmable logic 
blocks, and a hierarchy of reconfigurable interconnects that allow the blocks to 
be "wired together", like many logic gates that can be inter- wired in different con- 
figurations. 
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III. Working of Proposed System 

A. Data Acquisition 

Data acquisition is the process of collecting input data from input devices. So sig- 
natures are collected using android mobile phone whereas finger prints are col- 
lected from the F inger Print Sensor R3 05 . 

B. Image pre-processing 

Pre-processing stage is used to reduce the noise and normalized the images 
obtained from input devices. The pre-processing stage is enclosed by normaliza- 
tion, segmentation, filtering like processes. Binarization and thinning process 
are carried out over the finger print image in the pre-processing stage. 

C. F eature Extraction 

Feature extraction is the efficiency measure tool for the signature verification pro- 
cess. Proposed system, deals with the dynamic features and used function based 
features as signature is characterized in terms of time function. Dynamic features 
are extracted from signatures that are acquired in real time which make the signa- 
ture more unique. In proposed system, we are going to deal with the some fea- 
tures like thinning of images, perimeter, area, orientation, eccentricity etc. Pat- 
tern of interleaved ridges and valleys lines can be described by a fingerprint 
image. Minutia is the unique feature of the ridges. Minutia points occur at ridges 
bifurcation. Bifurcation is the process where a ridge split into two lines at spe- 
cific points. The feature extraction of fingerprint will consist of finding the ridge 
ending and ridge bifurcations from the input fingerprint images; begin each minu- 
tia described by its location and orientation. The final ridge structure will be used 
to generate feature vector know as minutiae, which will characterized the finger- 
print. This will be template formed by a list of minutiae and a list of number of 
ridges between each pair of minutiae, and it will be stored by the system. 

D. Classification 

In verification process, authentication of signature, fingerprint and key pattern 
are done. In these, the features of signature and finger print and key pattern which 
stored in dada based during enrolment stage are matched with test signature, fin- 
gerprint and key pattern. K nearest neighbours (KNN) classifier used for the clas- 
sification stage. The K-nearest-neighbour (KNN) algorithm measures the dis- 
tance between a query scenario and a set of scenarios in the data set. 

1. Distances: 

We can compute the distance between two scenarios using some distance func- 
tion d(x, y) where x, y are scenarios composed of N features, such that, x = 

(Xj xj and y = (y 1 yj. Two distance functions are discussed in 

this summary: 

Absolute distance measuring: Equation 1 
JV 

d A (x.y) =2>, - I 

1*1 

Euclidean distance measuring: Equation 2 
JV ~ 

d*Cx.y)=2, J**i-y*i 

t* 1 ^ 

Because the distance between two scenarios is dependent of the intervals, it is rec- 
ommended that resulting distances be scaled such that the arithmetic mean 
across the dataset is 0 and the standard deviation 1 . This can be accomplished by 
replacing the scalars with according to the following function: 



Equation 3 


Where x is the unsealed value, x is the arithmetic mean of feature x across the 
data set (Equation 4), a Or) is its standard deviation (see Equation 5), x’ and is the 
resulting scaled value. The arithmetic mean is defined as: 



Equation 4 

We can then compute the standard deviation as follows: 


1 « 

4 ® 

2. Distance functions (K-nearest-neighbour): 

As stated previously, we are only considering absolute (Equation 1) and Euclid- 
ean (Equation 2) distance functions d(x, y). However, we may choose to provide 


the original unsealed values, or use transforms them using the scaling function in 
Equation 3. Now that we have established a measure in which to determine the 
distance between two scenarios, we can simply pass through the data set, one sce- 
nario at a time, and compare it to the query scenario. We can represent our data set 

as a matrix D=N x Z 3 , containing P scenarios S 1 S P , where each scenario S 1 

contains N features S-jS/, S n ' } A vector o with length P of output values 

accompanies this matrix, listing the output value o 1 for each scenario s 1 . It should 
be noted that the vector o can also be seen as a column matrix; if multiple output 
values are desired, the width of the matrix may be expanded. 

KNN can be run in these steps: 

1 . Store the output values of the M nearest neighbors to query scenario q in vector 

r= {r , ,r M } by repeating the following loop times: 

a. Go to the next scenario Si in the data set, where i is the current iteration within 
the domain 

(1, ,P} 

b. Ifisnotsetor q < d(q,S l ):q «- d(.q,s l \t «- o l 

c. Loop until we reach the end of the data set (i.e. i = P). 

d. Store q into vector c and t into vector r. 

2. Calculate the arithmetic mean output across as follows: 



3. Return f as the output value for the query scenario q. 

IV. Implementation of Proposed System 

To establish the experimental setup make all the connections as shown in fig 4. 
Firstly environment connection is necessary to make the wireless connection for 
that purpose we are using Wi- Fi connection. Afterword move toward evaluation 
phase in which user must enrol him/her to the system by using signature and fin- 
gerprint. This information of the user stored in the data base in enrollment phase. 
Next in evaluation phase test signature, fingerprint and key pattern are compared 
with previously stored one. Proposed system setup works in real in three phases: 

A. Pre saving of segmentation on android 

B. User Registration 

C. User Evaluation 

A. Pre saving of segmentation on android: In gesture builder library emulator 
save the gesture file. It provide extra layer of security. 

B. User Registration: User draw the sign on the android phone it followed by fin- 
gerprint scan on the module R302. Both the input saves to the data base. Key pat- 
tern also generated by the user. All these three data are saved by user name on 
knowledge base. 

C. User Evaluation: As we know, authentication process consists of two phase, 
enrollment phase and verification phase. We finish enrollment phase through the 
user registration. Now whatever data we are needed for evaluation i.e signature, 
fingerprint and key pattern is already stored in data base. Here we compare previ- 
ously stored user data with newly enrolled if all the parameters matches then we 
get the proper authentication result. 

1. Signature process: 

For the signature, we are using android phone as a input. Wireless connection 
should be created between Android phone and Matlab. The gesture file which is 
created when user touches the screen and it gets stored in Matlab. Android pro- 
vides the Gesture Detector class for detecting common gestures. A Hidden 
Markov Modelling (HMM) is used with the signature process. Knn classifier is 
used for classification. K nearest neighbours is a simple algorithm that stores all 
available cases and classifies new cases based on a similarity measure. A case is 
classified by a majority vote of its neighbours, with the case being assigned to the 
class most common amongst its K nearest neighbours measured by a distance 
function. If K = 1 , then the case is simply assigned to the class of its nearest neigh- 
bour. 

2. Fingerprint process: 

For fingerprint in proposed system we are using finger print sensor module 
R305 . In terms of applications, there are two kin ds of fingerprint recognition sys- 
tems: verification and identification. In verification, the input is a query finger- 
print and an identity (ID), the system verifies whether the ID is consistent with 
the fingerprint. The output is an answer of yes or no. The output is a short list of 
fingerprints. Generally, fingerprint-matching algorithms have two steps: (1) 
align the fingerprints and (2) find the correspondences between two fingerprints. 
Dynamic timing wrapping is implemented on fingerprint for the feature extrac- 
tion. It involves extraction of minutia points from ridges. Dynamic time warping 
(DTW) is an algorithm for measuring similarity between two temporal 
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sequences which may vary in time. DTW is a method that calculates an optimal 
match between two given sequences (e.g. time series). 

• Two time series Q&C. 

Q=<hq 2 . q n 

C C [ 5 C 25 c n 

• Construct m*n matrix D with distances D tj =d(q i , c J ) 

• Warping path W is a contiguous set of matrix elements W k =( i,j ) k 

• Find : DTW ( Q,Q = min •'£ U’Jc 

In minutiae extraction identification systems minutiae considered are ridge 
bifurcations and terminations. Whereas in neighbourhood extraction list of minu- 
tiae neighbours is extracted for each minutia. The minutiae are ordered according 
to the increasing distance from the image centre and each list is ordered accord- 
ing to the increasing distance from the corresponding minutiae. The image centre 
is assumed to be the mass-centre of the detected minutiae. The minutiae list pro- 
vides information on the minutiae position and on the ridge/ valley behaviour 
between minutiae. For this reason, the line connecting each minutiae and one of 
its neighbours is traced, and the grey values extracted. 

The complete embedded system works in two phases of authentication i.e enroll- 
ment phase and evaluation phase. In enrollment phase signature, fingerprint and 
user name are stored on database. In evaluation phase, system compares the 
user's signature; fingerprint and key pattern with previously stored one and 
according to that evaluation phase give its output. This system gives the correct 
authentication when all three parameters are matched. If system found that any 
among three is not matching with the previous one then we are not getting any 
authentication. System display the message “match not found”. 

We can observe the result evaluation phase and enrollment phase in following fig- 
ures. As all setups are done, Wi- Fi connections are made and pre saving of seg- 
mentation on android are done the firstly our set up check whether serial port con- 
nections is done or not. For establishing serial port connection, correct com no 
should be placed as shown in fig 2. From fig 2, there is option for clearing data 
base through which user can clear data base if required. 


Finger and Signature Oe He rat ion 

C.0U EBW 

Add ys*ri 10 system Evaluate System 



LW Do- 

Fig.2. Serial port connection 



Fig.3 (a) Signature enrollment 



Fig. 3(b) Fingerprint enrollment 



Fig 3(c) Enrollment of user name 



Fig. 3(d) Enrollment phase result 



Fig 4 Authentication's result 

For enrolment, user must click on “Add user to system” through which user can 
enroll their signature and fingerprint as shown in fig 3(a) and fig (b) respectively. 
After enrolling signature and fingerprint, user can stored that data by their name 
as shown in Fig 3(c). Finally fig 3(d) shows the result of enrollment process. As 
the enrollment phase completed, user information will be stored in data base. 

In evaluation phase, classification is carried out between previously stored infor- 
mation with present one. Same steps are carried out for evaluation phase, same as 
shown in Fig. 3(a) and Fig. 3(b) and key pattern also must be entered. Finally 
authentication is displayed as shown in Fig 4. In following table we tabularized 
the result. As proposed system are used three parameters such as signature, fin- 
gerprint and key pattern. System authenticates the user only when all three 
parameters are matched. Table 1 represents the result of proposed system when 
all three parameters are matched. Table 1 shows the 7 user entry, all users are cor- 
rectly implements the system then system detect the correct user every time. 


Table 1 : All parameters are correctly enrol 


User 

Actual 

signature 

Actual 

fingerprint 

Key 

pattern 

System 

detected 

User 1 

User 1 

User 1 

1 

User 1 

User 2 

User 2 

User 2 

1 

User 2 

User 3 

User 3 

User 3 

1 

User 3 

User 4 

User 4 

User 4 

1 

User 4 

User 5 

User 5 

User 5 

1 

User 5 

User 6 

User 6 

User 6 

1 

User 6 

User 7 

User 7 

User 7 

1 

User 7 
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Table 2: Result obtained by varying fingerprint 


User 

Actual 

signature 

Actual 

fingerprint 

Key 

pattern 

System detected 

User 1 

User 1 

User 7 

1 

Match not found 

User 2 

User 2 

User 6 

1 

Match not found 

User 3 

User 3 

User 5 

1 

Match not found 

User 4 

User 4 

User 4 

1 

User 4 

User 5 

User 5 

User 3 

1 

Match not found 

User 6 

User 6 

User 2 

1 

Match not found 

User 7 

User 7 

User 1 

1 

Match not found 


Table 3: Result obtained by varying signature 


User 

Actual 

Actual 

Key 

System 


signature 

fingerprint 

pattern 

detected 

User 1 

User 7 

User 1 

1 

Match not found 

User 2 

User 6 

User 2 

1 

Match not found 

User 3 

User 5 

User 3 

1 

Match not found 

User 4 

User 4 

User 4 

1 

User 4 

User 5 

User 3 

User 5 

1 

Match not found 

User 6 

User 2 

User 6 

1 

Match not found 

User 7 

User 1 

User 7 

1 

Match not found 


Table 4: Result obtained by varying key pattern 


User 

Actual 

signature 

Actual 

fingerprint 

Key 

pattern 

System 

detected 

User 1 

User 1 

User 1 

0 

Match not found 

User 2 

User 2 

User 2 

0 

Match not found 

User 3 

User 3 

User 3 

0 

Match not found 

User 4 

User 4 

User 4 

0 

Match not found 

User 5 

User 5 

User 5 

0 

Match not found 

User 6 

User 6 

User 6 

0 

Match not found 

User 7 

User 7 

User 7 

0 

Match not found 


Table 2shows the result of the proposed system when we enroll fingerprint incor- 
rectly and remaining two factors correctly. From table 2, system detects the user 
4 correctly and in remaining cases it display that match not found. Table 3 also 
show the result with maximum accuracy in which we obtained the result by vary- 
ing signature factor. 


Table 5: summarized Result 


User 

Actual 

Signature 

Actual 

fingerprint 

Key 

pattern 

System detected 

User 1 

User 1 

User 2 

0 

Match not found 

User 2 

User 2 

User 2 

1 

User 2 

User 3 

User 1 

User 3 

1 

Match not found 

User 4 

User 4 

User 4 

1 

User 5 

User 5 

User 5 

User 5 

0 

Match not found 

User 6 

User 6 

User 6 

1 

User 6 

User 7 

User 5 

User 3 

1 

Match not found 


Table 4 shows the result of proposed system by varying third factor i.e key pat- 
tern. Here we use 1 and 0 for representing correct key pattern and wrong key pat- 
tern. If result table shows 0 in key pattern column it means key pattern is not 
matched whereas when 1 is displayed in result table it means key pattern is 
matched. Table 5 shows the summary of all above result table. We can find the 
accuracy of system by computing the accuracy of each table. The accuracy of the 
system is given by the ratio of Number of correctly classified outputs to the total 
outputs. 

Accuracy = Number of correctly classified outputs / Total outputs 

F or Table 2 : Accuracy = 7/7= 1 . It means accuracy is 1 00% . 

We can calculate the accuracy for all tables by using same formula mention 
above. From the above accuracy calculations we can say that proposed system is 
more accurate than previous one. So that we can say proposed system having low- 
est EER and lowest FTC. 


IV. Conclusion 

This paper proposed a real time authentication framework using multi-model 
biometrics. It consists of the embedded system to verify the signatures, finger- 
print and key pattern for authentication of user. Proposed framework provides a 
strong user authentication solution with the extra layer of security. When a high 
level of security is needed, it is necessary that you combine more than one 
authentication factors. This work implements such a biometric system which con- 
sists of 'Physiological trait (fingerprint) as well as Behavioural trait (signature) of 
biometrics. The complete embedded system works in two phases of authentica- 
tion i.e enrollment phase and evaluation phase. In enrollment phase signature, fin- 
gerprint and user name are stored on database. In evaluation phase, system com- 
pares the user signature, fingerprint and key pattern with previously stored one. 
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